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(54) ATM switching 

(57) A communication system comprising a hub slot 
adapted to receive any one of a plurality of hub cards tor 
receiving and transmitting data cells; a plurality of uni- 
versal card slots; a plurality of Interface cards Insertable 
Into any one of said plurality of universal card slots for 
receiving Incoming ones of said data ceils containing 
data and transmitting outgoing ones of said data cells 
containing data; en add bus having respective data links 
connected between Individual ones of said universal 
card slots and said hub dot tor receiving said outgoing 
ones of said data ceils from said plurality of Interface 
canto and transmitting said outgoing ones of said data 
ceils to said hub slot; a drop bus having a single data 
link connected between all of said universal card slots 
and said hub slot for transmitting said Incoming ones of 
said data ceils from said hub slot to said plurality of 
interface cards; and means within each of said interface 
cards for filtering said Incoming ones of said data cells 
from saw drop bus and thereby routing said data ceils to 
an appropriate one of more of said plurality of interface 
cards. 
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Destitution 

rtaldofthe Invention 

5 (0001] This Invention rotates fn general to digital communication systems, and more particularly to a novel switching 
system using asynchronous transfer mode (ATM) transmission and switching of Information. 

Background of the Invention 

w [0002] The emergence of high-speed asynchronous transfer mode (ATM) communications Is a recent result of the 
diverse demands now being made on enterprise backbone networks. Early enterprise networks were dominated by 
voice traffic with only a relatively small amount of circuit bandwidth devoted to data and other appfioatkms. More 
recently, a range of new applications has evolved resulting In significant changes to existing backbone networks. High- 
bandwidth video telephony and video conferencing, for example, are rapidly becoming essential requirements In digital 

is communication systems. SMIany the bandwidth requirements for LAN (Local Area Network) Interconnection across 
multiple sites Is also Increasing as established prior art LAN systems such as Ethernet 01 and Token Ring are upgraded 
to meat the demands of fester communication and more sophisticated processing. 

[0009] For example, Rbre Distributed Data Interlace (FDD)) LANs operating at 100 Mbps are presently being 
deployed while even higher bit rates LAN types are emerging as a resuft of text-based personal computers being 

so replaced by multi-media work stations and associated servers. Typically, multi-media work stations and their associated 
servers support document architectures that comprise not only text but also high resolution still Images and moving 
Images with sound. Thus, Instead of Inter-site LAN traffic being dominated by file transfers of textual Information as in 
the prior art, LAN file transfers In newer systems are migrating towards higher volume, high bit-rate mixed-media traffic. 
[0004] The combined effect of such developments has necessitated the development of a more flexible method for 

sb the allocation of transmission bandwidth m order to efficiently utilize inter-site leased circuits associated with enterprise 
networks. 

[0005] The developments discussed above are not limited to private networks, but are occurring in pubfc carriers 
as wen. 

[0006] In order to meet these new demands In private and public digital communication systems, an international 
ai standard operating mode has been developed for use with broaotoand integrated services digital networks (BISON) 
based on the asynchronous transfer mode (ATM) of transmission and switching. The aim of the ATM protocol Is to pro- 
vide a more flexible facility for the transmission and switching of mixed-media traffic comprising data, voice, etlD and 
moving Images and video. Traditionally, constant bit rate traffic such as voice has been transmitted and switched using 
p re-assigned time slots, whereas data Is normally transmttted in the form of variable length frames which are mufti- 
35 ptexed together on a statistical basis. According to the ATM protocol, transmission and switching Is performed on fixed- 
sized unite referred to as "cells*. Cells from different sources (eg. voice, date, video, etc.). are multiplexed together on 
a statistical basis for transmission purposes. 

[0007] Each standard ATM ceil is 53 bytes in length, comprising a 48-byte information field (also referred to as the 
"pay load*), and a five-byte header containing routing and other fields. 

40 [0008] Like packet and frame switching, ATM operates on a virtual callteonnection basis. This means that prior to 
any user Information calls being sent, a virtual connection Is first placed through the network. During this phase, a vir- 
tual connection Identifier (VCI) Is assigned to the call at each Interexchange link along the route. The assigned Identifier, 
however, has only local significance to a fink end changes from one link to the next as the ceils relating to a connection 
pass therethrough. This means, therefore, that the routing information carried In each cell header can be relatively 

« small. 

[0009] In particular, each Incoming link/port has associated therewith a routing table that contains the correspond- 
ing output llnkfcort and a new VCI to replace the Incoming VCI for the subsequent finkftort The routing of cells in both 
directions along a predetermined route Is therefore extremely fast as It involves only a simple look-up operation. As a 
result, ceils from each link can be switched independently and at very high rates. This allows parallel switch archltec- 

50 teres to be used and high-speed circuits (ie. In the glgabft-per-eecond range), each operating at Its maximum capacity. 
[001 0] In practice, the VCI Is made up of two sub-fields: a virtual path Identifier (VPl) and a virtual channel Identifier 
(VCI). The VPl field relates to statically assigned connections whereas the VCI field relates to dynamically assigned 
connections. Routing can be performed using one or the other, or a combination of the VPl and VCI eubfiefds. For 
example, a virtual path may be set up through the network on a semi-permanent basis (by network management) 

55 between each pair of network endpolnte. The cells relating to multiple (la concurrent) cells between these end points 
are then multfeiaxed together and then routed along the same assigned path. In this example, therefore, the routing of 
ceils within the network Is performed using the VPl field and the VCI field would be used at the end point to relate cells 
to a particular can. 
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[0011 J The ATM reference model defines three protocol layer*, as follows: (1 ) Am adaptation layer which overlies 
the (2) ATM layer which overflss the (3) physical layer. 

{0012] The ATM adaptation layer (AAL) provides a range of alternative service classes tor performing an adaptation 
function between the class of service provided to the user (ag. for the transport of data frames between two LANs), and 
6 the cell-based service provided by the ATM layer. 

[0013] The ATM layer provides the required mu&lpfaxlng of cells relating to different connections Into a single 
stream of cells, and the subsequent demultiplexing of the cell streams. The ATM layer also effects the required rout- 
ingfretaying of ceils based on the VPI and/or VCI fields. 

(0014] The physical layer Interfaces with the particular transmission medium which carries the actual ceils (eg., 

io fibre optic, coaxial cable, etc), and may be frnplementsd via a number of different communication technologies depend- 
ing on the type of transmission being used (eg. pleslochronous or synchronous). For the former, the transmitter estab- 
lishes a frame structure over the bit/byte stream that exactly matches the ATM celL The receiver then processes the 
Incoming byte stream on a byte-by-byte basis until a vaQd 5-byte cell header Is formed. The incoming byte stream fe 
then processed on these fixed cell boundaries. In the case of a synchronous link (e.g. OC3/STM1 ), the frame psyioad 

13 field Is not a multiple of the ceo size ami hence the ceil boundaries wOl change from one frame to the next With this type 
of link, therefore, a pointer In the overhead channels is used to Identify the start of the first cell boundary In the pay load 
field while ceil delineation b performed based on the HEC byte (discussed in greater detail below). 
[0015] As discussed above, the ATM layer performs all of the functions relating to the routing and mult^laxing of 
cells over virtual connections which may be semi-permanent or set 14) on demand. For the latter, a signalling protocol 

20 Is frnplemented which Is similar to that used wfth (SON. 

{0018] There are two different header formats for standard ATM celts commonly referred to as UNI and NNI. Each 
format incorporates a VPI field as the first byte. However, for the format used over a user-network access link intended 
for use by user devices that generate and receive ceRs directly; the four most significant bits of the first header byte are 
replaced by a generic flow control (GFC) field that has only local significance over the Rnk and Is included to allow cells 

23 to be allocated different priorities. This field is not present within the network, however, and Instead the VPI field Is 
extended across the entire byte. 

[0017] The second byte of the header comprises a first nibble which is an extension of the VPI field. Thus, tor the 
format used over a user-network access link, the VPI field Is eight bite, whereas within the network the VPI field is twelve 
bits. The least significant four bits in the second byte of header Information comprises a first portion of the VCI field. The 

3D third byte of the header continues the VCI field and the first four most significant bits of the fourth byte of the header 
complete the VCI field Thus; the VCI field in a standard ATM header consists of sixteen bite. The four least significant 
bits of the fourth header byte include (1) a psyioad type (FT) field which Is used to enable cells relating to the C and M 
planes associated with the ATM reference model to be differentiated from cells containing user information, and (2) a 
cell-loss priority (CLP) bit The CLP bit Is used to allow a user to Indicate those calls associated with a connection which 

as should be discarded first This is useful because an ATM network operates by multiplexing on a statistical baste so that 
It Is possfcte for cell buffers to overflow within an exchange. 

[001*1 Finally, a header error control (HBC) field is provided as a variation of an eight-bit cydte redundancy check 
(CRC) polynomial for detecting errors in the header. If the CRC polynomial foils, the ceil Is discarded. However, for sin- 
gie-ttt errors, hardware may be provided to correct the error based on Information from the HEC field. 
40 [0010] There are a number of areas In the design of existing ATM-based communications systems where substan- 
tial improvements may be made in signal routing efficiency, diagnostic support and hardware amplification. 
[0Q2O] Firstly, it is desirable to provide a system which can flexibly accommodate a variable number of interlace cir- 
cuits per switch fabric Interface, depending on Interface card bandwidth. Prior art systems have been provided with fixed 
bandwidth for each Interface card Within such systema 
45 [0021 ] Secondly, while the virtual connection identifier (VCI) may be used to establish routing of a ceH from link-to- 
link on a point-to-point basis or "shared" from one point to several destinations (I.e. polnt-to-murti point), ft can only do 
so at the expense of costly and complex circuitry. Similarly, only a rudimentary level of cell priority queuing Is possible 
using standard ATM ceil headers. Abo, according to many prior art systems, intarehelf communication has been via 
parallel busses which are of inherently low speed and low bandwidth. Therefore, there Is a need for inexpensive 
so enhanced routing capability of ATM ceils both inter-node and infra-node within such communications systems. 

[0022] Thirdly; since ceil streams In an ATM communication system are essentially point-to-point and terminate In 
queuing points, there Is normally no need to maintain synchronous dining throughout the switching fabric However, 
since some interface cards require a standard timing reference, ft is desirable to maintain system tfrnlng in such a sys- 
tem. The standard method for maintaining intra-node system synchronization In an asynchronous serial link (eg. Inter- 
as shelf link}, is to run a synchronous timing link throughout the system. However, such systems suffer from Jitter transfer 
problems resulting from synchronously regenerated taring signals due to Instability of chaining phase lockecHoops 
(PLL). 

[0023] Alternatively, some prior art systems maintain synchronization by providing a dedicated timing wire from the 
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system synchronization unit (SSU) to all toning destinations. This effectively restricts the location of the SSU to a pro- 
determined slot in the system which la specfflcaOy wrined to receive ft. 

(0024 Accordingly; there la a need to maintain system synchronization without extra timing wires and without suf- 
fering from loss of sync and other problems Inherent In prior art PLL synchronization systems. 
5 [0026] Fourthly In prior art systems debug access to the operafbig software h the system is pravidad by a spedal 
software load with debugging coda built In, and a de d icated hardware debug port must be provided onto which debug 
equipment must be attached In order to gain access to the debug software. It is desirable to provide a system in which 
the debug software Is always in place and In which the development system support communfcaflone are integrated into 
the ATM fabric. 

w [0026] Finally, it Is desirable to provide system redundancy for Improved reliability of critical system functions. 
[0027] Other opportunities exist for the improvement of ATM communications systems design, such as in the areas 
of control communications, queue servicing algorithms, node synchronization architecture, etc. 

Summary of the invention 

13 

[0028] According to the present Invention, an improved ATM communications system Is provided the design of 
which has been directed by the desire to address the prior art problems discussed above. 
[0029] Firefly, the maximum utiftzetflon of the switching core is achieved in the system of the present Invention by 
providing a variable nurrfcer of Universal Card Slots (UCS) per Inter Shelf Link (ISL), depending on the interface card 
a> bandwkfih. For example, a large number of tow-epeed UCS cards may be provided in one embodiment, while fewer 
high-speed UCS cards may be provided In another embodiment, and in each case switching core efficiency may be 
optima8y maintained. 

[0030] Secondly, with respect to the problem of routing ATM celts within the communication system, according to 
the present invention a plurality of overhead bytes are pre-pended to the standard 63 byte ATM caO In order to ferifftste 
a cost-efficient ceil routing on a point-to-point basis or a poinMofnulQpoint basis within the system, with celt priority queu- 
ing, simplified egress statistics gathering, and error detection across the pre-pended bytes and the four bytas of the 
ATM header (with the HEC field omitted). 

[0091] Thirdly, with respect to the problem of maintaining system synchronization, according to the present inven- 
tion an 8 Khz timing signal is embedded in Ordered Sets (drscussed in greater detail below) which are transported over 

3a the iSLs and can appear anywhere In a "Superceir framing structure (the concept of a "Superceil "is discussed in 
greater detail below). Therefore, the timing signal is independent of any serial data clock, as contrasted wfth prior art 
synchronous systems, No special per reducing circuitry or wiring Is required, and the timing source and SSU can be 
located anywhere within the switching fabric to which the ATM cells may be directed, in contrast with prior art synchro- 
nization systems using a dedicated tirrfng wire. 

35 [0032] Fourthly, with respect to development system support, according to the present invention an integrated real- 
One development system is provided wherein the debug software is permanently installed in the system and in which 
the development system support communications are integrated into the ATM fabric, such that the prior art requirement 
for spedal debug equipment at a customer site is minimized, if not entirely eOnlnatBd. 
[0088] Finally, the system architecture of the present invention allows tor switch fabric redundancy. 

40 

Brtef PescrMton of tho Drawings 

[0084] A detailed description of the preferred embodiment is provided herein below with reference to the following 
drawings, In which: 

45 

Figure 1 1s a block diagram of an exemplary digital communication system Implementing the method and apparatus 
of the present invention; 

Figure 2 is a block diagram showing the cell switching core In greater detail connected to an exemplary perfcherai 
so sheff of the system Illustrated in Figure 1; 

Figure 3 is a diagram of the modified ATM ceil format for point-to-point communication, according to a firs em bod- 
Vnent of the Invention; 

85 Figure 4 is a dtegam showing the modified ATM caO format for point-to-multipolrrt communication, according to a 
second embodrnent of the invention; 

Figure 6 shows a card address format according to the preferred embodrnent; 
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Figure 6 is a Mock diagram of an Interface circuit for connection to a universal card dot and to external signal car- 
rying mecBa, Including drcutrry tor generating and filtering the proprietary ATM header data according to the pre- 
feared embodiment; 

5 Figure 7 Is a block diagram of a switching ASIC used on a hub card of the peripheral shelf shown in Figure 2; 

Figure 8 is a block diagram showing a cell queuing core of the ASIC shown In Figure 7: 

Figure 9 te a block diagram of a standard Interface ASIC In the interface circuit of Figure 6 for receiving and trans- 
io mttttng formatted ATM cells to and from the switching fabric; 

Figure 1 0 Is a flowchart showing operation of a receive filter in the interface circuit of Figure 9; 

Figure 1 lb a flowchart showing details of a multicasting ceil sorting procedure in the flowchart of Figure 1 0; 

13 

Figure 12 Is a flowchart showing operation of a first filter sorting algorithm in the flowchart of Figure 10; 

Figure 13 Is a flowchart showing operation of a second filter sorting algorithm in the flowchart of Figure 10; 

so Figure 14 Is a functional schematic diadem of an tntershelf link according to the present Invention; and 

Figure 1 5 Is a block diagram showing distribution of timing information through the communication system of Fig- 
ures 1 and 2, according to the present Invention. 

25 Petal led Description of the Preferred Embedment 

[0085] With reference to Figure 1, a block diagram is provided of a switching architecture for Implementing the 
method and apparatus of the present Invention in accordance with one embodiment The system comprises a cell 
switching core 1 connected to a plurality of Interface card access or peripheral shelves 3A, 3B, etc, via respective 800 

so Mbps fnter-eheff links (ISL) 5. In the present disclosure, the terms "access shelf and "peripheral shelf will be used 
interchangeably throughout In a multi-shelf access arrangement such as shown with reference to blocks 3D and 3E, a 
further ISL 5A may be provided directly between the access shelves. Furthermore, in some system "stand-alone" con- 
figurations, a single Interface card peripheral shelf may be provided without the requirement of a ceil switching core 1 . 
Alternatively, a muftVehetf access arrangement such as shown with reference to blocks 30 and 3E may be provided In 

55 stand-alone configuration with a switching circuit incorporated directly into each peripheral shetf. 

[0036] All externa! Interfaces (ag. OC-3, video, FDD!, etc) terminate on interface cards located in twelve universal 
card state (UCS) located on each peripheral shelf 3B, 3C and/or 3D, as discussed to greater detail below with reference 
to Figure 2. In the mutt^sherf access arrangement 3D end 3 E, up to ninety-six (96) universal card slots may be provided 
for each Inter-shelf link (ISL) 5. Furthermore, according to the present Invention the number of UCS interface cards 

40 sharing an ISL may be made variable depending on the Interface card bandwidth. For example, a large number of low- 
speed UCS cards may be provided In one emborftnent (eg. 3D), while fewer high-speed UCS cards may be provided 
in another embodiment (eg. 33). This flexibility provides better utilization of the cell switching core 1 and provides more 
control of statistical gain. 

[0037] Each 800 Mbps ISL 5 is adapted to transmit ATM cells between the cell switching core 1 and associated 
4S ones of the access shelves 3A, 3B, etc. using either electrical or optical, full duplex, fibre channel (FC-o and FC-1 on V) 
Interfaces, in a well known manner. 

[0036] Turning to Figure 2, the cell switching core 1 1s shown functionally as providing inter-shelf ceil switching tor 
respective ones of the access shelves 3A, SB, eta The switching core 1 uses an Input cell filtering and output queuing 
architecture to Implement a ceil space switch (Le. ceils can be switched to any output from any Input), in the preferred 

so embodiment, the switching core 1 can range from 2 to 258 »SL ports per system. Therefore, the maximum switching 
capacity Is 256 ISL/eystem x 800 Mbpa/ISL = 204.8 Gbpsteystem. The ceB switching core 1 1 ncor p o ra te s a plurality of 
dual switching cards (such as 1A, 1B and 1C shown in Figure 16). Each such dual switching card has access to the 
switching core 1, and provides two output ISLs 6 for connecting up to two peripheral shelves 3A, 3B, etc. 
[0030] In Figure 2, a representative peripheral shelf 3C Is shown connected to the switching core 1 via respective 

ss ISLs 6. As discussed above, the peripheral shetf 3C Incorporated twelve universal card slots (UCS) for receiving respec- 
tive interface cards 21 for rnptemenfing aB Interface (la between the switching fabric and the outside world via I/O Inter- 
lacing to optical, coax, or other physical medium), control and resource functions. For the purpose of describing the 
present invention, the terms Interface card" and UCS card" may be used interchangeably. Each peripheral shelf, such 
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as the shelf 3C shown In Figure 2, includes two special purpose hub cards (only one hifc card 29 being shown tor ease 
of Illustration), which form part of the entire switching fabric. Hie switching fabric of the preferred embodiment Is fufly 
duplicated for redundancy. Accordingly, one hub card is provided for each naff of the fully duplicated switching tabic. 
The hub card & multiplexes and concentrates cells from multiple interface cards 21 onto the 800 Mbps fntershetf links 

5 (ISLb 6) con nected to the switching core 1 . Each UCS housing an Interface card 21 has a 200 Mbps Interface to the hub 
card 23, designated as an add bus 2& As discussed above, the hub card 23 terminates an tSL 5 from the switching 
corn 1 and drives a further 800 Mbps shared bus on the backplane, this bus being designated as the drop bus 27 from 
which the UCS cards 21 Miter received ATM cells. The hub card 23 also includes a loop-back circuft 28 which Is normally 
provided for diagnostic purposes. However, In a stand-alone configuration of the access or peripheral shelf 3C, the loop- 

hi back 29 can be used for directing the 800 Mbps data which is concentrated from the add bus 25 back to the 800 Mbps 
drop bus 27. 

[0040] The system has a duplicated ATM switching fabric for fault tolerance. The major components of the switching 
fabric are the hii> cards 23, switching shetM, and the ISL cables 5. The interface cards 21 put customer data onto both 
fabrics. 

15 [0041] According to a further possible stand alone configuration of the peripheral shelf 3C, the switching core 1 may 
be effectively incorporated into the peripheral shelf itself where two or more such peripheral shelves with internal switch 
core are connected together (such as peripheral shelves 3D and 3E connected via ISL 6A in Figure 1 ). The number of 
ports connected to the switching core 1 can be made flextrfe (eg. a large number of low bandwidth ports may be added) 
to fully utilize available bandwidth. 
£? - [0042} Interfaces which require greater than 200Mbps of system switching capacity are interlaced directty with the 
Switching core 1 via associated ISLa 5 (e.g. high speed interface 3A in Figure 1). 
[0043] As discussed in greater detail below, according to the present invention data on each of the aforementioned ' 
800 Mbps Ante (is. ISLs 6, drop busses 27, etc) is assembled as a succession of "Superceits", each comprising an 
Ordered Set (la 32 bit longword aligned data structure for control information) followed by 128 BO-byta proprietary ATM 
23 ceils. The use of these supercede results In straightforward cell delineation and supports a simple protocol tor relaying 
' various types of system level status Information. 

[0044] In operation, for amplified data flow In the aforementioned stand-alone configuration of the peripheral shelf 
3C (i.e. without routing through the switching core 1), each UCS or interface card 21 provides the appropriate One ter- 
mination, performs AAL/ATM layer processing of recerVed data, adds additional routing Information to the ATM cell to 
so create a formatted cell header in accordance with the principles of the present invention, and sends the formatted ceils 
to the hub card 23 over the 200 Mbps add bus 25. As discussed in greater detail below, the formatted cell of the present 
invention has seven additional overhead bytes pre-pended to the standard 53 byte ATM eel I, to form a 60 byte formatted 

[0046] For BOO Mbps peripheral shelves 3C, the hub card 23 multiplexes and concentrates the formatted cells from 

35 the individual UCS cards 21 onto an 800 Mbps ceil stream which is looped back via the aforementioned embedded 
switching core (not shown), or via the loop-back 29 to all of the UCS slots on the common drop bus 27. For other than 
800 Mbps peripheral shelves, the bop-back function may or may not be provided internally on the hub card 23. Each 
Interface card 21 filters the cells from the BOO Mbps drop bus 27 using the routing information which was added to the 
cell header, queues the ceils, performs AAL/ATM layer processing on the transmitted data and drives the associated 

40 line Interfaces via the Interface card 21 . 

pXMfi] For data flow through a larger node which uses switching core T, the system operates In an identical manner 
as discussed above in connection with a small node except that Instead of looping back the data to the drop bus 27 
through the embedded switching core or loop-beck 29 of hub card 23, the 800 Mbpe cell stream Is encoded within hub 
card 23 with an 8B10B code (as per fibre channel FC-1 standard) and converted to a 1 Gbeud serial stream which fa 

45 then sent to the swHchtng core 1 via ISL 5. The ceil stream Is received by switching core 1 and the celts within the ISL 

6 are routed to the proper ISL outputs of switching core 1. 
* [0047] The hub card 23 on the appropriate peripheral shelf (e.g. shelf 3C, etc), receives the celt stream from the 
switching core 1 via ISL 5 and in response drives the BOO Mbps shared drop bus 27 on the backplane of the peripheral 
shelf. Then, as discussed above in connection with the smaller node interconnection, each UCS housing an interface 

so card 21 fitters the ceils from the 800 Mbps drop bus 27 using the routing information that was added to the ceil header, 
queues the cells, performs AAL/ATM layer processing on the transmitted data and drives the associated One interlace 
via the interface card 21 . 

[0048] The configuration of add bus 26 and drop bus 27 results in a "star bus" topology which provides unique 
advantages over the prior art in the implementation of a high speed communication system. It Is known chat high speed 
55 data transmission is most easily accomplished with point-to-point transmission lines. Therefore, by splitting the add bus 
25 Into a plurality of point-to-point links in the present Invention, significant advances are made over prior art conven- 
tional pdnt^D-muftyolnt architectures (e.g. using multi-party bi-directional busses). Such prior art systems suffer from 
the following problems: 
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- low impedance and discontinuous transmission fries due to card loading 
difficult line terminations 

hfch 8 peed busses requiring parallel terminations that consume significant power 

- the effective speed at which the busses can operate Is limited by factors such as arbitration for bus mastership. 

a 

[0049] The polnt-co-polnt communication provided by the add bus 25 In the star-bus topology off the present Inven- 
tion overcomes these problems. 

[0050] In the "drop" direction (I.e. drop bus 27) since all the UCS cards 21 are required to receive all of the Incoming 
data cells, a unidirectional bus 27 Is utilized. Since the bus 27 Is unidirectional, the topology of the present Invention 

10 benefits from simple transmission One termination requirements. 

[0061 ] According to the present Invention, proprietary overhead Information is added to the standard 53 byte ATM 
call in order to assist In the routing of the oefls through the switching fabric The csfl format of the present Invention 1b 
used on all links between various cards of the system. Thte Includes the links from the UCS cards to the hub card 23, 
links to and from the switching core 1 , and from the hub card 23 to the UCS cards 21 . 

rs [0052] As discussed above, In accordance with the preferred embodiment, seven bytes are pre-pended to the 
standard 53 byte ATM cell In order to form a 60 byte formatted ceil. The additional header Information Is used to 
uniquely address any "port" on any UCS housing an interface card 21 and to identify the priority of the attached ATM 
cell. The additional header information Is also used to support a mum-casting capability where the address field Identi- 
fies a pjoup of UCS Interface ports. Use of the additional header information pre-pended to the standard ATM cell 

so allows for improved ceil routing over prior art ATM-based switching systems. Unused bits in the header may be used for 
other control functions (eg. providing signalling information at the discretion of software). 

[0063] As discussed In greater detail below, there are two cell types defined by the additional header Information 
according to the principles of the present invention, as follows: (1) point-to-point; and (2) polnt-to-multl point 
[0054] Sendi rig cells to a specific card withi n the system requires thai the eel la be routed to th b drop bus 27 to which 
28 the particular UCS Interface card 21 is connected. The card then must filter the cells destined for It from the remaining 
cells present on the drop bus 27. 

[0055] When a cell is addressed to a particular UCS Interlace card 21, the drop bus 27 that the particular card lis- 
tens" to is referred to as a terminal bus" (Le. the data on the bus Is not routed to a subsequent bus). If, on the other 
hand, a cell is addressed to a card that is pert of the switching fabric, the bus that Is "listened" to by that card may be 
3D an intermediate bus whose data to routed to other buses, in accordance with the present Invention, the definition of the 
routing of the cell through the fabric Is Identical In both cases. As discussed in greater detail below with reference to 
Figure 9, drcuftry is provided In each UCS interface card 21 for filtering the cells on the monitored bus In order to rec- 
ognize correct ceils destined for the particular card. 

[0056] Figure 3 Illustrates a formatted ATM ceil in accordance with the present Invention, for implementing point-to- 
ss point communication. The fields pre-pended to the standard ATM cell are defined In Table A below. 

[0057] It should be noted that for all field definitions throughout this disclosure, bits are assumed to be transported 
In the order of left to right and top to bottom. In multi-bit fields, the most significant bit is transported first 



TABLE A 



Field Name 


Description 


MT 


When this bit is low the cell is considered to be an empty celL The CRC value 
Is checked to determine if any bit errors occurred, otherwise the ceD is dis- 
carded without any processing being done. 


Pt-R 


indicates addressing Is either for a point-to-point or for a polnt-to-muitipolnt 
connection. "1 " = point-to-point; V = point-to-multipoim. 


NCT 


Newbridge Cell Type. These bits ere used by the Interface ASIC to determine 
If the cell is part of the normal data stream or If It should be routed to the inter 
neJ control or RDS queues. 


RFU 


Reserved for Future Use. These bits should be set to "0" to guarantee compat- 
ibility with future Vnplementafions which may use them. 


Priority 


Indicates cell priority. Supports 8 priorities. TOO" = Lowest Priority; "111" = 
Highest Priority. 


AAL5 


identifies this cell as being part of an AAL5 frame. 
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TABLE A (continued} 



Field Name 


Due ration 


Source Port 


Indicates the cells ingress port Range: 1 Zero Is illegal. 


Stage 1 /Stage 2/Stage 3 Address 


These fields each allow the selection of one output out of sixteen from a switch- 
ing sheff, with the capability of having three stages of switching shelf. This per- 
mits the construction of exceptionally targe switching systems. 


Card Address 


This field uniquely Identffies a destination element within an ISL 


Egress Connection Identifier 


This held b set on Ingress on the ceD relay cards and Identifies the connection 
at the epess point It te used for performing address translation and statistics 
gathering on egress, rt permits the transmission of an identifier which is easier 
and more economical to use when updating statistics than the entire VP/VC 
held of the ATM address. 


Port 


Used by multi-port Interlace cards to address a port (from up to sixteen). In 
cells sent to the swlttftoglabric tWs field te RRJ. 


NCC 


Newbridge Communications Channel This 4-bft wiD provide tor Information 
exchange between cell processing dements In a switch node. Preliminary use 
may Include bits for local switch congestion Indication. 


CRC-8 


Cyclic Redundancy Check. Provides error detection for the combined New- 
bridge and ATM header. The polynomial used Is x 8 +x 2 +x 1 +1 . 



2s [0058] As indicated in Table A, the card address flew Is used to select a destination element within an ISL One pos- 
sible definition of this field is shown in Figure 5, although other addressing definitions are possible. 



30 
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Fgg Egg 

Range* 



Hit 
nnacef 



P— e rlptlan ~~~ 

inetyeocarfl.i«Hbe" 
Mncu auuliei 



10 



M 



Location* 



bis 



0000 -lO Shelf *1, Interface ASK «1 
0001 - UCS Shelf*?, famine ASIC f! 
00H>- UCS Shelf #3, interface ASIC #1 
0011 -UCS Shetf*4, Interface ASIC *1 
0100- UCS Shelf «5. Interface ASK #1 
0101 - UCS Shelf #6. Interface ASJC #1 
01 10 -UCS Shelf «7, Interface ASIC tl 
Oil I - UCS Shelf «B. Interface ASIC «1 
1OD0-UCS Shelf #1, Interface ASIC tt 
1001 - UCS Shelf #2, Interface ASIC *2 

1010- UCS Shelf «9. Interface ASIC «2 

1011 - ALL Hob Cards 

1 100 - UCS Shelf #1 , Interface ASIC #3 

1 101 - UCS Shdf #2, Interface ASIC *3 
11 JO. 

1111. 



In thu format, Ranges 0-7 correspond to the UCS carts in 
thrived. Ranees 1-1S are tliahUy more comobceied. 
Ibe Locitioo * Identifks a map* kxmwo withro a ~ 



It iscooodod in ih& 

UCS Bm i n (RnfB 0.T0. 12.11.14) 
3 3 10 



[ 



) 



1-16. 



Stot 

Slot #0-13 identify (be pulpbcjil 
HUB Ranges: (Range 11) 

3 2 10 

r Shelf* | xnr | 
Shelf #0-7 identify the chained shelf nnmSen 1-8. 
X/Y Mendfiei the fabric wkhO- X. I »Y 
— Ransn ]S) 

3 2 10 



RPU. set « 0 until defined. 

Stipe #0-2 identifies the switching stage 1*3, with (stags 
43 m anRFU) 

Xnr jctemiteibcfahdcwithQ»X 1»Y. 



55 [OG59] Transmitting celts which are part of a pdrrt-to-muUpoint connection requires that the cell be routed to every 
drop bus 27 which has a card that is part of the muW-cast pjoup. The call must also contain a mutt-cast identifier that 
each card checks to determine ff the card is part of the predetermined muffl-cast group for the cell. This group oan then 
be used to determine which ports of the UCS cards are to use the cefl (ie. which interface cards 21 are to receive the 
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data). 

(0060] The cefl format of a point-to-muJtipolnt caO Is given in Figure 4. The field definitions are provided in Table O 
below. 

B TABLE C 



Held Name 


Description 


MT 


When this Mt is iow the cell Is considered to be an empty ceil. The CRC 
value is checked to determine If any bit errors occurred, otherwise the ceil 
Is discarded without any processing being done. 


Pt-Pt 


Indicates addressing is either for a point-to-point or for a polnt-to-murrjpolnt 
connection. "1 " = point-to-point; "0" = polm-to-multipoint 


NCT 


Newbridge Cell Type. These bits are used by the Interface ASIC to deter- 
mine If the cell Is part of the normal data stream or if it should be routed to 
the Internal control or RDS queues. 


RFU 


Reserved tor Future Use, These bits should be set to "0" to guarantee 
cornpatibUfty wfth future foplernentations which may use them. 


Prtorlty 


Indicates ceD priority. Supports 8 priorities. "000" = Lowest Priority; "111"= 
Highest Priority 


RFU 


Reserved tor Future Use. These bits should be set to "0" to guarantee 
compatibility with future Implementations which may use them. 


Switch Sheff Output Bitmap 


A multicast ceil may be routed to multiple drop busses. This is accom- 
pushed by bit mapping the output ports of the switching shelf that the cell 
is to take. 


MUuicasE La roup uonnecoon icennner 


1 ufa neio is set on m grass on me ceu re say earn ana loeniines a system 
wide unique Multicast Group. 


AALfi 


Identifies this cell as being part of an AALB frame. 


Source Fort 


Indicates the cefTs ingress port Range: 1 ..3. Zero is illegal. 


NCC 


Newbridge Communications Channel. This 4-bft will provide for Informa- 
tion exchange between cell processing elements In a switch node. Prelim- 
inary use may Include bns for local switch congestion Indication. 


CRC-8 


Cyclic Redundancy Check. Provides error detection tor the combined 
Newbridge and ATM header. The polynomial used Is x 8 +x 2 +xVl . 



40 

[0061] The cell header describes a number of different cell types Including data cells, control ceils and RDS cells. 
This allows control and RDS communications to be carried In -band within the data switching fabric. Many systems use 
an out-of-band control channel which restricts the control card to a specific location In the system. Allowing the control 
and RDS communications to be carried In-band Within the data switching fabric allows scalability to very high band- 
45 wlolhs and Increases reflabilrty. In-band communications means that no special hardware or software is required at the 
local site and debugging can be done remotely. 

[0062] Turning now to Figure 6, the functional blocks of a representative UCS Interface card 21 are Illustrated. The 
Illustrative example shown In Figure B b an OC-S/STM-1 interlace card for connection to a peripheral shelf 3B, 3C, 3D 
or 3E (Figure 1). Interface cards which are appropriate tor implementing a high speed Interface or an 800 Mbps inter- 
so face may be devised using similar functional elements as shown In Figure 6. 

[0003] As discussed above, the basic function of the OC3/STM-1 UCS Interface card 21 is to transport ATM cell 
data between the switching fabric and the SONET/SDH network fink. The blocks required to perform this function may 
be Identified as follows: 

55 (1) Control/status block 71; 

(2) Synchronization block 73; 

(3) Backplane interface block 76; 

(4) ATM block 76; 
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(5) SONET/STM-1 Mock 77; and 

(6) Transport medium and Interface block 78. 

[0064] Tha comroVstatus Wock 71 provides coordination of Interlace functions and establishes node control com- 

5 munfcation via the backplane Interface block 75. 

(0065] Tha synchronization block 73 accepts anoVbr generates a system synchronization reference, as discussed 
In greater detail below. This block generates liming signals required for all functional blocks of tha UCS card 21 , Includ- 
ing the provision of fining signals whereby the SONET/STM-1 transmission meets predetermined Jitter and accuracy 
requirements ff a Synchronization Unit (SU) Is located on the UCS card 21. 

yo [0066] The backplane Interface Wock 75 processes the specially formatted ATM cafe (la ATM oalls having adcB- 
tional pre-pended bytes) that are transmitted to and from the switch! ng fabric, and provides data Integrity checking, con- 
nectivity checking and conversion of cells between the specially formatted ATM oeils and standard ATM oalls. Tha 
functional requirements for this block are discussed In greater detail below with reference to Figure 9. 
[0067] The ATM block 76 processes the ATM cells that pass between the backplane Interface block 75 and the 

is SONET/STM-1 block 77, Inducing VPI/VCI mapping, usage parameter control (UPC) policing, and per VPI/VCI statis- 
tics gathering. ATM block 76 comprises an ingress ATM logic block 76C, an egress ATM logic block 7BA, an Ingress 
UPC 76B and Ingress m i croprocessor context memory interlace 7B0 

[0088] The Ingress ATM logic block 76C or Ingress Cell Controller (abbreviated as ICC) provides the following ATM 
layer functionality: (1) VPI/VCI address compression, (2) ceti counters. (3) QAM control cell process! n& (4) QAM cell 
» extraction, and (5) prepensing of the seven header octets to the ATM ceil (Figures 3 and 4). A 64Kx1 6 SRAM 1 702 pro- 
vides the ICC with per connection QAM functionality end VPI/VCI compression tables. 

[0069] There is a global bit located In the ICC 76C which is programmed upon Infttaflzation to signal an Internal 
address compression block whether the link is UNI or NNL When the link is UNI, an Bbit VPI and 16btt VCI is com- 
pressed to I2btta When the link is NNI, a 12bft VPI and 16blt VCi Is compressed to I2bfts (referred to herein as ICI). 
26 [0070] The 12bit resultant ICI allows the OC-3 Card to support up to 4K of connections using any VPI and a VCI 
within the range of 0 to 409& 

[0071] When a cell is received, the VPI Is used to Index a VP Table. The result la a 16 bit word which determines If 
this VPI has been enabled and whether ft* a VPC or VCC. rf the connection is VPC, the VP Table entry also contains 
a 1 2 bit ICI. if the connection is VCC, the VP Table contains a VC Table pointer and a VCI Mask. The VC Table pointer 
so points to one of 17 2K VC Sub Tables. Tha VCI Mask is used to determine how many of the VCI bits wHI be used to 
Index the VC Sub Table. This mask must be either 11 or 12. The OC-3 doesn't support any other mask selections. The 
unused VCI bits are compared to zero, if they contain non-zero values, the can Is considered Invalid and the appropriate 
actions occur. Otherwise, the VC Sub Table entry contains an ICI for the VC connection. 

[0072] Once Id has been generated, it Is used to point Into the 1 CCa context memory 760. A bit is checked to verity 
as validity of the connection, if it lent a valid connection , ICI Is Ignored, the Ingress UPC 76B Is advised that It has an InvaDd 
cell and the connection's VPI/VCI values are stored Into the IOC memory's Invalid Table, if the connection Is enabled. 
ICI is passed to the Ingress UPC 76B. 

[0073] The memory accessed by Ingress UPC 76B Is the 64Kx32 memory 76F residing on the Ingress UPPs host 
port This memory provides the Ingress UPC with; UPC Information, per connection statistics, NATM header octets (to. 
40 Internal Newbridge ATM ceil formats in accordance with Tables A and D), and VPI/VCI translation bytes. 

[0074] The Context Table within memory 76F contains 4K data structu res. Each data stnichj re represents Informa- 
tion for a VP or VC switching connection. The UPC table contains 1 .5x4K (6K) dam structures which each represent the 
necessary in fo rmati on for a bucket 

[0075] NATM header registers are provided as memory locations in 76F which contain the seven octets represent- 
45 Ing the NATM header. These fields are prepensed to the beginning of the cell header for use throughout the switching 
fabric Included within these fields are port addresses, ECI (Egress Connection Identifier), MQI (Multicast Group Iden- 
tifier) etc 

[0076] The SONET (Synchronous Optical Neiworky5TM-1 block 77 adapts the ATM ceils received from and trans- 
mitted to the OC-3/STM-1 physical layer, and provides overhead processing for the section, One and path layers, it also 

50 provides One (egress) and diagnostic (Ingress) loop back capabRrty. More particularly, the SONET/STM-1 Interface 
block 77 provides both an S-blt 10.44Mhz and a serial l55Mhz access to the Transport Medium Interface 7B and 8-bit 
25Mhz access to me ATM block 76. Multiple serial Interlaces are also provided for an optional NNI module. 
[0077] The Interface block 77 also provides (either directly or via the NNI module) full access to SONET/STM-1 
framing information and provides four-deep receive and transmit FIFOs (not shown) for the ATM layer Interface 76. It 

55 also delineates ATM ceils and provides HEC checking and correction. 

[0078] The transport medium Interface 78 provides optical (or coax) interfaces, dock recovery and data timing 
between an optical medium 70 such as fibre optic cable (or coax medium 70A). The transport medium Interface 78 also 
provides electro-optical conversions required to pass the ATM cells to and from the optical OC-3/STM-1 link. The func- 
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ttonal requlnenisrrta tor transport medium Interface block 76 are discussed m greater Malt below, 
[0079] For signal flow in the egress direction, the backplane interface block 76 monitors the type of formatted ATM 
cefls, and atecrtminates between data, RDS, control and empty cetle. The cell type is determined by Kb NOT and MT 
bite (see Figures 3 and 4). 

5 [0060] Data celts are passed by the backptane interface 75 to the ATM Week 78. The destination address of each 
active cell la checked before the cell is passed to the ATM block. The egress ATM logic 76A straps off the seven format- 
ted ATM cell header octets from each active cell before passing it to the interface 77. The seven formatted ATM cell 
header octets are generated and added to each cell received In the inpess direction by inpess ATM logic 76C, before 
transmission to the switching fabric, as discussed in greater detail below. 

w [0061] The RDS and control cells are not transmitted to the ATM block 76. Instead, these celts are stored for use 
by the control/status block 71 . In the Ingress direction, FIDS and control calls are created by the control processor 71 A 
and Inserted Into the Ingress ATM cell stream for transmission through the switching fabric 
[0062] Empty cells passing In the egress direction through backplane interface 75 are discarded, tn the Ingress 
direction, a nibble Is added to the eel I to indicate the start of a cell, ff there are no cells to be transmitted to the swftchlng 

ib fabric, the link remains idle. 

[0083] In the egress direction, multicast cells are received and used to look up an enable bit in a multicast look-up 
table (discussed In greater below with reference to Figure 10). If a match occurs, the cell Is accepted; otherwise, it is 
discarded. Furthermore, point-to-point cells In the egress direction are received and compared to a pair of fitter registers 
(discussed in Teeter detail below with reference to Rgures 12 and 13). An exact match is required for the celts to be 

so accepted. Otherwise, the cells are discarded. 

[0064] Cells passing in the egress direction are placed In one of four priority queues. CLP discard can be enabled 
and Is performed when a programmable discard threshold is matched or exceeded. These queues also provide forward 
congestion notification if enabled through the PTI bit field of the ATM header. The ASIC Incorporated into backplane 
interface 75 (discussed In greater detail below with reference to Figure 0) provides statistics for the number of cells 

sb arriving (1 6 Wt); the number of cells discarded CLP=0 (1 6 bit); the number of ceils discarded with CLP= 1 (1 6 bit) and 
the number of cells arriving congested (1 B bit). Status flags are also available for full and empty queues; discard state 
and congested state. 

[0065] The backplane interface 75 also provides a variety of maintenance features. Firstly, by defining Invalid point- 
to-point Altera for the cell comparison, the control processor 71 A Is capable of detecting Incorrect destination addresses 
ao of ceils passing through the backplane interface 75 in the egress direction. Also, a loop back function may be provided 
to the loop ingress path entering the backplane Interface block 75 to the egress data path exiting the backplane block. 
This provides a means to test the ATM block 76 and SONET7STM-1 block 77 during power up diagnostics. 
[0066] it is necessary for the control/status microprocessor 71 A to access the memory 1702 In order to initialize 
and "rip down" connections, instead of using a dual port memory architecture, the ICC 76C directly controls the mam- 
as ory. Whenever the microprocessor 71 A requires access to the memory, ft tells the ICC what it wants to do, and the ICC 
executes the necessary Instructions on behalf of the microprocessor 71 A. This way the ICC 76C knows when It lent 
using the memory during a cell time and can allot that time for the microprocessor Interlace 1 703. 
[0067] In addHSon, the backplane interface 75 is capable of declaring certain alarm conditions. As discussed above, 
redundancy Is provided by means of duplicated drop busses In each peripheral shelf 3A, 3C, etc. Each of the two drop 
40 busses provides a loss of clock Indicator for the epess cells coming from the switching fabric. A hardware Indicator is 
active when no transition has been detected on the Interface clock for 140 nanoseconds. This time is derived from 14 
clock cycles of the C 100 M clock utilized by the ASIC discussed below with reference to Figure 6. The UCS card soft- 
ware monitors ATM receive clock failures for the redundant ATM switch fabrics. The UCS card software provides an 
alarm indication when this failure Is alarmed on the active ATM Interlace. 
45 [0068] The UCS card hardware also monitors the level of the four queues for the egress celts received from the 
switching fabric, in the event that the buffer fills, the event is counted and aggregated as a statistic value. 
[0088] According to the preferred embodiment, ATM ceil processing Is performed by means of a pair of application 
specific I ntegrate d circuits (ASICs) within the switching core 1 , hub cards 23 and UCS cards housing Interface cards 21 . 
The first ASIC Is shown in Figure 7 for performing a dual purpose switching function. In one application, the circuit of 
so Figure 7 Is used in hub cards 23 of the access shelves SB, eta, for multiplexing the 200 Mbps date on add bus 26 into 
a single 800 Mbps can stream for application to Inter-ehelf links 5. In the second application, the circuit of Figure 7 Is 
utilized In the switching core 1 to filter (i.e. switch) a pair of 800 Mbps Input ceil streams Into a single 800 Mbps output 
stream. The 800 Mbps output stream can then be shared by multiple additional ASICs of the form shown In Figure 7 to 
provide filtering (l.a switching) of the same 800 Mbps output link from multiple Input links in the switching fabric. 
5$ [0090] In the "multiplex mode" of operation, six input processors 81 receive respective 200Mbps signal streams 
from respective UCS cards housing interface circuits 21 . Thus, by using two of the switching and muftflaxJng circuits of 
Figure 7, the desired functionality of the hub card 23 may be implemented for concentrating twelve 200 Mbps cell 
streams carried by the add bus 26 Into a single shared BOO Mbps output stream. Each 200 Mbps input date stream is 
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processed via a respective processor 81 for call delineation and for CRC checking. The add bus Rnte 26 from the UC3 
cards to the hub card 23 consist of a data nibble (La. 4 bits) and clock signal only, so that cell delineation may be per- 
formed based on a staple algorithm which recognizes calls being preceded by a unique start-o*cell nibble, or other 
suitable technique. 

5 [0091 ] Each of the format converterftnuttiplaxBrs 83 gathers the three 200 Mbps streams output from processor 81 
and converts them to a single 800 Mbps Input stream tor further processing by a cefl queuing core 85. The cell queuing 
core 86 Is discussed In greater detail below with reference to Figure 8 A pair of muftipJexsrs 68 respectively select one 
of either the 800 Mbps Input (switching mode) or the three 200 Mbps cefl streams output from format ccnvertetfmuW- 
plexer 83 (multiplex mode) for input to the cell queuing core B& Thus, the system provides sufficient flexibility to have 

io one of the 800 Mbps Inputs to the cell queuing core 85 configured as three 200 Mbps Inputs <le. muttfclaxer mode) while 
the cither 800 Mbps Input is configured as a direct 800 Mbps Input (ie. switch mode). 

[0092] Slot monitor FIFO 87 provides a microprocessor Interface to "monitor" a specific 200 Mbps input or a specific 
800 Mbps Input from the muftfclexers 88. The circuit of Figure 7 captures a cefl from the appropriate Input link when so 
directed via the micro pr oce ss or port The microprocessor then drectty reads the full 60 byte formatted ATM ceH from 
io FIFO 87. 

[0093] The cell queuing core 85 Is shown in greater detail with refere nce to Figure 8 comprising 600 Mbps process- 
ing Mocks 91 which perform dock detection, link termination, CRC checking and oel) filtering functions. When an 800 
Mbps input stream to the cell queuing core 85 is eourced from three 200 Mbps Inputs (la multiplex mode) the cell filter- 
ing function of processors 8 1 is typically disabled. Trie allows ail ceils In the input streams to be queued. As per the 200 
» Mbps inputs, each 800 Mbps Input can be enabled or disabled from having their respective ceils enter queue memory 
93. 

[0094] Memory manager 95 controls four cell queues wBhln memory 93, for providing a total of 256 cells of queuing 
space which can be fiextofy allocated between the four queues. Memory manager 96 operated on the data contained 
within the four queues to process each cell in accordance with ail aspects of the ATM ceO header, Including CLP discard 

20 and PTI congestion notification. 

[0095] An arbitration control 97 provides Information on the current state of the cefl queues to an external arbiter 
(not shown). When multiple circuits share the same 800 Mbps output link, the external arbiter is required to decide 
which circuit source is the next cefl and at which priority. The arbitration control 97 provides the external arbiter with all 
of the necessary information required to implement any queue service algorithm which can be downloaded and is 

so reconfigurable at any tine. 

[0098] The output formatter 98 creates a formatted 800 Mbps link {as well as inserting the appropriate priority oeil 
when so instructed by the external arbiter), in the form of a "supercell", as discussed in greater detail below. 
[0097] Insertion FIFO 99 is provided to allow control and real-time development system (RDS) ceils to be transmit- 
ted onto the 800 Mbps output link Details of the RDS functionality are provided below. As discussed in greater detail 

ss below; the interface circuit of Figure 9 provides a standard RDS and control interface to the local microprocessor. The 
circuit of Figure 7 provides an interface to the circuit of Figure 9 to transmit these controVRDS cells onto the 800 Mbps 
output port The 800 Mbps input processors 91 contain a plurality of registers which are used for cell filtering. Specffl- 
caity, point-to-point and polnt-to-muftipolnt cell filtering is accomplished using internal 'mask* and Value* registers 
against which input values may be compared and must be matched (or alternatively masked) in order for a polnt-to- 

40 point or poinMo-muftlpolnt cell to enter an Internal queue from the 800 Mbps Interface. In this regard, ail ceil filtering In 
the system of the present invention is conducted via pattern matching. 

[00981 Before turning to Figure 9, a brief description Is provided herein of the RDS (Real-Time Development Sys- 
tem) functionary of the ATM switching system according to the present invention. 

[0099] RDS Is used extensively in the system according to the present Invention, to develop and debug software. 
45 Debugging can take piece in a range of environments from a development lab to a customer site where it can be used 
on live equipment without Impacting service or customer data. As discussed below, the RDS function of the present 
Invention operates in an event mode and to a command mode. 

[D1 00] RDS events are embedded In the actual software at development time and In most cases are not subse- 
quently removed for two reasons: (1 ) events can assist fn tracing subsequent problems, (2) taking them out would affect 
so the real time execution of code, which may have real functional effects, even though the code has been designed so as 
to not be sensitive to execution speed. 

[0101] An RDS event Is simply a set of writes to an RDS event port, embedded In the software at significant soft- 
ware Interfaces and points of interest The data that Is written to the port includes an event Identifier and a variable 
length sequence of data bytes that define what software event Is occurring. It is similar in concept to putting a "print" 
£5 statement within the software to indicate that this portion of code is executing and using the print data to Indicate exactly 
what is happening. 

[0102] In the ATM switching system, RDS events are generated by nearly ail of the processors In the system and 
the event data Is placed on the ATM switching fabric along with control messaging and customer data. To reduce the 
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amount of bandwidth consumed on tho ATM ewBching fabric* the ASIC (Figure 9) contains a hardware filter that can 
discard HDS events based on the event Wenffiter. In normal operation of the ATM switching system according to the 
present invention , all of the events generated by all the processors In the system ere discarded using the hardware fitter 
of the ASIC In Figure 9. 

a (01 08] The events can be enabled onto the switching fabric by changing the state of these hardware fitters. This can 
be done selectively for each interface ASIC (Figure 9) In the system and for each of the 256 everrte that the ASIC sup- 
ports. TWs allows someone with knowledge of the RDS events In the system to enable selective events to aid In Diag- 
nosis of problems. 

[01 04] RDS events can be displayed on a VT1 00 terminal or a workstation. Generally an additional card Is Installed 
w Into the system for filtering the RDS events off the switching fabric and formatting them for display on the VT1 00 or work- 
station. The ATM cells that carry the RDS event data Indicate the source address and using this knowledge, the event 
Identifier and the event data, text can be formatted and displayed on the VT100 terminal or workstation that corre- 
sponds to the event which has occurred In the software. The control card of the ATM switching system Is also capable 
of filtering the RDS event cells and formatting them for display. This aOows remote debugging of a live system since 
is access to this control card is available across the network. 

[0105] Since the events are left In the code, the priorffles of code design are to keep the size of the event cods to a 
minimum, to keep the processing load of event generation to a minimum, and to ensure that the proper events are within 
the code to allow the diagnosis of problems and visibility Into the operation of the system. 

(0106] As discussed above with reference to Figures 3 and 4, the contents of the header field of the call are com- 
so pleteiy programmable, Including the ATM header VCI/VPI fiekte. The CRC header protection field Is calculated and 
inserted into the ceO header automatically, and a two byte field is provided for a RDS source csidress to be used by the 
receM ng end to Identify the source of the cell. As discussed in greater detail below, an address mapped set of registers 
that store the fields of the cell header are provided In the Interface ASIC of Figure 9. This allows nrwdJfteations to be 
made to portions of the header field without rewriting the whole header each time a small change b needed, (ag. prl- 
3B ority changes). With control over the ATM VCI/VPI fields, event cells can be routed directly out of the switching system 
88 true ATM cells to a destination outside the node, without having to be relayed by the control complex or an RDS card. 
This feature allows event cells to be transmitted to a remote debug location directly, assuming that a connection to the 
network is available. However, It should be noted that the pre-pended bytes of Figures 3 and 4 (Including the NCT bits) 
are lost when the ceil exits the node, but this information Is not usually needed if the receiving entity Is expecting only 
so RDS event ceils. 

[01 07] In command mode, RDS Is used to actively debug processors in the system by allowing source and assem- 
bly level debugging as well as memory read and write operations. According to this mode of operation, a host generates 
one or more RDS command cells, each comprising a command identifier and a variable length Identifier for defining the 
command, to a target system. The target system responds by returning to the host an RDS event cell containing the 
sb results. The commands are very ernpie (eg. reed memory at a given address, write memory to a given address with 
provided data. Identify target processor type, return values of target processor registers, eta) Using these simple com- 
mand operations, the host system ie capable of constructing en advanced functionality such as source level debugging, 
assembly level debugging, breakpoint insertion and stack tracing etc 

[0108] When an RDS command cell Is tittered off of the backplane by an interface ASIC (ie. the ASIC shown in Fig- 
40 ure 0), It queues the celt and generates a high priority non-maskable Interrupt to the target processor on the associated 
card. The use of a non-maskable Interrupt allows the RDS system in the command mode to reliably Interrupt the target 
processor so that other Interrupt handlers on the card can even be debugged using the RDS. 
[01 00] Since both RDS command and RDS event cells conform to standard ATM cefi format In accordance with the 
present Invention, these cells can be transmitted across an ATM network so that an RDS host processor can remotely 
45 debug a target processor by operating In a remote mode. The ingress Interface circuitry (Figure 6) of the system con- 
taining the target processor causes the ceil type (NCT) to be set to RDS Command for RDS command cads arriving on 
the VP/VC (Figure 8). 

[01 1 0] Turning now to Figure 9, a functional block diapram is provided of the Interface ASIC which performs ATM 
backplane functions required tor any card to Interface with the switching fabric. As such, the circuit of Figure 9 Is 
so Intended for any interface, hub or switching card that transmits and receives ATM cells through the system, such as the 
UCS Interface card 21 discussed above wtth reference to Figure 6. 

[011 1] m order to send cells Into the switching fabric, the circuit of Figure 9 provides a receive flnk interface 1 00 in 
the form of an externally timed interface for formatted ATM cells to be transmitted on the add bus 26. This receive fink 
Interface 100 operates at a maximum of 400 Mbps, although the maximum add bus rate Is 200 Mbps, as discussed 
55 above. 

[01 1 2] Trie completely formatted ATM cells received from a UCS (or other) card via receive link 100, are applied to 
me add bus 26 via add bus interface/header protector 101 along with an Inserted CRC-8 byte in the ATM HEC field. As 
discussed above with reference to Figure 6, the UCS card assembles completely formatted cells using the header fields 
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shown in Rguros 3 and 4, except tor the CRC-8 byte. The CRC-e byte covers the seven octet (to. byte) overhead as 
well as the four remaining octets of the standard ATM header. 

(0113) Control cells and RDS cells can be applied to tfw add bus 25 through FIFOs 102 and 104 which are acces- 
sible through a m i cro p roc es sor interface 1 06. 

6 {0114] The circuit of Figure 8 provides a separate interface for each of the redundant drop buses 27. For each drop 
bus 27, the circuit monitors for a loss of dock and for CRC errors on all cells, via drop bus f raming/CRC check circuits 
1 08. The signals output from circuits 10S are muttfplexBd at 1 10 and applied to a receive cell fitter 112. 
[0115] The received cells from the active tabrtc are then filtered vto receive ceO fitter 112 to determine which celts 
are addressed to the associated Interface circuit 21 . Control/RDS celts and user data oells are filtered using the prede- 

io fined Card Address field (Le. the fourth byte In the polnt-to-pol nt cell format of Figure 3) to fadDtete Interface card redun- 
dancy, as discussed In greater detail below. Mum-cast cells are verified against entries In an external, 64K connection, 
multi-cast look-up table 1 1 6, also discussed in greater detail below. 

[0116] Turning to Figure 10, the receive cell filtering process executed by fitter 1 12, Is shown in detail Upon receipt 
of a formatted ATM ceO (step 1 24), empty ceib are Identified and discarded (step 126). Identfficstion of empty cells is 
to accomplished by checking the MT bit to the first octet of die formatted ATM header. Next, the Pt-Pt bit is queried to 
determine whether the ATM ceD is formatted for point-to-point or point-to-muJt^olnt addressing (step 1 2B). Address fil- 
tering Is then soft into mufti-cast and point-to-point portions of Figure 1 0. 

[0117] For multi-cast cells, the Mum-cast Group Identifier field Is used to look up an enable bit in a multicast look- 
up table (MOT) stored within an external RAM 1 1 8, discussed in greater detail below, if a match occurs (step 1 30), the 
20 cell Is accepted. Otherwise, the cell Is discarded (step 127). Accepted ceils are then sorted according to the Newbridge 
Ceil Type (NOT) field in the header (step 132). 

[0113] Turning briefly to Figure 1 1 1n conjunction with Table E below, the multicast cell sorting step 132 is shown 
In greater detail 



TABLE E 



NCT 


Cell Type 


00 


User data 


01 


Control 


10 


RDS Command 


11 


RDS Event 
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[0119] Upon receipt of a multi-cast cell (step 134), the NCT bits are analyzed to Identify an RDS Command, User 
Data, and Control or RDS Event (step 136). In response, sorting is continued on the basis of the cell being Identified as 
either an RDS CeD (step 138), a Control Cell (step 140) or a User Data Ceil (step 142). 

[0120] Returning to Figure 10, the identified RDS CeDs, Control Cells and User Data Cells are then accepted by the 

40 fitter 112, as represented by steps 144> 146 and 143. 

[0121 ] For point-to-point cells, the Card Address field of the expanded ATM header Is compared against the con- 
tents of two Internal fitter registers, hereafter referred to as F1 and F2. An exact match is required against the fitter reg- 
ister contents before a call Is deemed to have passed the filter function. Cells which do not match F1 or F2 are 
discarded (steps 150, 152 and 127). 

45 [0122] Control cells can be required to match F1 , F2. or either Fl or F2 before being accepted. User Data Cells 
pass through an Identical stage. This allows the Control Cells to be filtered off of one address, for example, the physical 
card adotess, and the User Data Cells to be filtered off of other addresses, for example, the physical card adctess of 
the redundant card. This also allows the User Data Cells (and/or the Control Cells) to be filtered off of either F1 or F2. 
This permits cells addressed to either card of a redundant pair to be accepted by both, RDS Cells are accepted only of 

50 the match F1. 

[0123] Details of the sorting and filtering procedure for F1 and F2 matched point-to-poim cells are shown In Figures 
12 and 13, respectively. 

[0124] Once a point-to-point cell has matched F1 (step 1 60) it Is then sorted by the expanded ATM header informa- 
tion (step 154). Wtth reference to Figure 12, upon receipt of the point-to-point (PP) cell (step 156), the Newbridge Cell 
55 Type b Identified using the criteria set forth in Table E, above (step 1 56). RDS Command Cells are accepted (step 160). 
Control Cells and RDS Event Cells are accepted if the Control Filter Select field (CFSJ1 :0]) In an Internal filter select 
register of filter 1 12 is programmed to accept F1 passed cells. The CFS bftfiekJ Is shown In Table F, below. Control and 
RDS Events cells win therefore be accepted If the CFS Wt field Is "10" or '1 1" (steps 1 62 and 164). 
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TABLE F 



5 



10 



CFS{1:0) 


Ml HItsr Selected 


00 


Undefined 


01 


Fitter 2 


10 


Filter 1 


11 


Fitter 1 orFllter2 



[0125] User Data Cells are accepted 9 the User Filter Select field (UF5[1 :0J) In the niter Select Register Is pro- 
grammed to accept F1 passed cells (steps 1S6 end 188). The UFS bit field to shown below In TkbteG. User Data Cefls 
is will theiefore be accepted* me UPS bttfl^ 

F1 or F1+F2, then theceH to discarded (step 170). 



TABLE G 



UFB(1:0) 


Cell Filter Selected 


00 


Undefined 


01 


Filter 2 


10 


Fitter 1 


11 


Filter 1 or Filter 2 



[0128] Once a point-to-point (PP) cell has matched F2 (step 152), ft Is then sorted by the expanded ATM header 
ao Information (stop 171). With reference to Figure 13, upon receipt of the point-to-point (PP) cell (step 172), the New- 
bridge Cell Type Is Identified using the criteria set forth In Table E, above (step 1 74). RDS command cells are discarded 
(step 1 76). Control Cells and RDS Event Cells are accepted If the Control Filter Select field (CFS[1 .-0]) In the Internal 
filter select register of filter 1 1 2 Is programmed to accept F2 passed cells. The CFS bit field Is shown hi Table F v above. 
Control and RDS Events ceOs wffl therefore be accepted If the CFS bitfield b "01" or *1 1 ' (steps 178 and 180). 
55 [0127] User Data Cells are accepted if the User Fitter Select field (UFS[1 :0]) In the Fitter Select Register Is pro- 
grammed to accept F2 passed cells (steps 1 82 and 1 64). The UFS bit field Is shown below in Table G, above. User Data 
Cells will therefore be accepted tf the UFS bit field to *0V or " 1 1 ".If either a Control Cell or a User Data Ceil falls to pass 
efther F2 or FH-F2. men the cell Is discarded (slap 176). 

[0126] The interlace ASIC of Figure 9 stores multicast lookup tables In the same external RAM 1 16 that Is used for 
40 queue memory. The first 2Kx32 block of memory, from address 0 to 600 hex, Is reserved for this purpose. The lookup 
tables are used when a multicast cell arrives, to determine if the mufficast pjoup is destined for the particular card. To 
accomplish this, the 16-bit Multicast Group Identifier In the expanded ATM header of the cell Is used to address a single 
bit of the mufficast block of external memory. The 18-bft Identifier Is translated Into an 1 1-bit address to access the 2K 
block of external memory, and a S*lt identifier to select which bit of the 32-bit wide data word to choose. This bit, a 
45 yes/ho indicator of the validly of the multicast cell to this ASIC, Is used when processing incoming cefls. A "0" in the 
memory location indicates that the multicast ceD b valid, and a "1* Indicates that the multicast ceil is Invalid. When no 
external RAM 1 16 Is used (which may occur In bypass mode), the data pins of the external RAM controller 1 16 may be 
tied to a logic high (eg. "1") level, so that all multicast ceils outside the internal lookup range will be discarded. Alterna- 
tively, the data pins may be wired to present a logic low (te. "0") value when a multicast "read" is performed, so that all 
so cefls are acc ept ed. 

[0126] An Internal lookup feature is supplied tor the use of an Interlace ASIC without external RAM 1 1 6. The upper 
most 32 bits of the external memory mufficast block are transparently mapped to an interna) 32-btt memory. This ena- 
bles a subset of the multicast capabilities for cards that have no external RAM. User access of the 32-bit Internal field 
Is transparent; It is accessed through the microprocessor as If It were in external RAM. The 32-bfts of external RAM are 
55 permanently mapped over. 

[0130] Receive cells which have been filtered by Receive Cefl Filter 112 are then sent to the designated receive 
queues via queue manager 114. User data cells are queued in external memory 1 1 6, through a 1 .6 Gbps memory inter- 
face 1 18. The queue manager 114 of the preferred embodknent supports up to 4096 ceils of externa) storage. RDS 
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command cells are sent to me RDS call FIFOs 1 04 tor access through the processor port 108. Control ceils and RDS 
event cells are cent to the Internal sixteen cell FIF0 102 for access through the processor port 1 06. The operation and 
flow of RDS command cells, control celts and RDS event cells discussed In greater detail below. 
[0131] Receive queuing to transmit fink Interface 120 can be bypassed (Le, queue manager 114 can be disabled 
5 for receive ceBs). This aiiows the circuit of Figure 0 to function as a receive caB fitter for an externa) cefl queuing device. 
However, the RDS and control cells are nonetheless sent to the Internal FIFOs 102 and 104, If eo enabled. 
[&1321 Finally, the receive cells are sent out the transmit link interface 120, under control from an external clock 
(TXUNKCLK). 

[0133] The circuit of Figure 9 services the priority "3 - queue via queue manager 1 14 If it contains a oell, and then 
to the priority '2* queue end so on down to priority V. However, this feature may be over-ridden via the transmtt Dnk arbi- 
tration port 1 22. This port provides an indication of a cell arrival, with the cell priority, to the aforementioned external 
arbiter (not shown), in return, the external arbiter can force the circuit of Figure 8 to service a given priority queue for 
the next ceO, regardless of whether a higher priority queue Is currently non-empty. 

PTI34J The BOO Mbps Inter-Shelf Link (ISL 6) is the common mechanism for connecting all component shelves 
is together in the system of the present invention. The ISL is a high-speed serial flnk which uses the lower layers of me 
Fiber Channel specification to transfer digital signals between access shelves 3A, 3b\ ett, In a muffl-eheff access 
arrangement, and between switching core 1 and the access shelves 3A, 3B, etc. As discussed above, each hub card 
23 generates proprietary 'supercells* tor transmission along the ISLs 5 and drop bus 27 using Fbre Channel technol- 
ogy. Speofftaafly, the output formatter 88 (Figure a) of the ewftohing ASIC ceil queuing core 86 (Figure 7), generates 
20 eupeicelts in accordance with the concept of an Ordered Set, as dscussed In greater detail below. A representative ISL 
6 is shown In Figure 14 comprising a high speed paratte! to serial converter (TX-151), equalization circuitry 153, con- 
nectorization 166, physical transport media (eg. shielded pair copper wires or optical ffeer), receiver coupling 168, ter- 
mination 167, and a high speed serial to parallel converter (RX 168). The 88/1 0B code requires a 1 Gbaud line rate In 
order to support the 800 Mbps data rate on the fink. The physical transport media for the Ftoer Channel interface can 
25 be etther electrical or optical. 

[0135] One of the features of the 8EV10B encoring scheme te the abiKy to communicate special command charac- 
ters over the serial flnk. The K2&5 code is a particularly special command character in that it le used by the receiver 
158 to establish byte and word synchronization. Additionally the K2ft£ character Is used within the system of the 
present Invention for cell delineation and optionally for the transport of the 8 Khz system synchronization signal, as dis- 
ss cussed in greater detail below. 

[0138] The Fiber Channel specification Introduces the concept of en Ordered Set. An Ordered Set (OS) Is a four 
byte grouping, composed of the K28.5 character and three additional data bytes. Ordered Sets can be sent over the ISL 
6 by asserting a special signal on the Fiber Channel transmit device 1 61 , and their presence Is detected at the receiver 
168 by the assertion of a OS indication BipjmL An ordered set Is defined as shown in Table H. 

35 



TABfiKH 


rst3i- 

BU24 


Bit 23- 
BU16 


BU8 ! 


Bb7- 

KtO | 



40 



I os Type 
I Bitfield 


mi 

Special Character 


Dnvc & Scan 
Byte (SOS only) 


m | 



[0137] The second byte is always the K28.5 special character The nret byte Is a fteW of 8 bits tor encoding the var- 
ious OS types. Since an OS may be considered to signal an event or statue condition, each condition Is assigned one 
bit in the field (as shown In Tfetole 0 which is set to Indicate the relevant event or status condition, allowing a single OS 
55 to encode numerous types of OS 'events'. These conditions are not necessarily mutually exclusive - for Instance, an 
OS with a first byte equal to 06H would Indicate both an SOS and an STOS. 
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TABLE I 



Bits 7-3 


Bit 2 


Bit 1 


6ft 0 


RFU 


STOS 


ETOS 


SOS 


Reserved For Future Use 


System Uming QS 


Extracted lining QS 


SuperceflOa 



[0138] The Drive & Scan Held Is only used If the OS Type Bltfleltf s SOS bit b asserted Otherwise, It Is treated as 
a "dont caw field. Upon reception of an SOS, the switching ASIC of Figures 7 and 8 will latch the Drtvaa Scan byte 
In an Internal register of the 800 Mbps input process ore 91 . For transmission of an SOS, the output formatter 88 (Figure 
8) derives the value of Its Drive & Scan byte from an internal register. This provides out-cf-band communication from 
one switching ASIC through the Fiber Channel to the next downstream switching ASIC. 

101 39] An Idle Ordered Set is defined by aU bits In the OS Type Bitfield having the value 0. RFU bits are set to zero 
by default 

{0140] The Inter^Shelf Link makes use of Fiber Channel technology and the notion of a •superce^ to aid In cell 
delineation. As Indicated above, a supercell consists of a Supercell Ordered Set (SOS) followed by 1 28 60-toyte propri- 
etary ATM cells. The Supercell format Is shown below In Table J. 

ratal 





Bit Number 


Word 


15 8 


7 0 


IT" 


05 Type Bitfield 


mj 


-r~ 

2 to 
3.841 


Drive kt, sen byte 


RFU 


128 x 60 Byte Newbridge ecus 



[0141] Supefteils are used on the Ftoer Channel ISLs 5 and the local drop busses 27, as well as Internally In cer- 
tain cards. The presence of any Ordered Sot is always Indicated by some sort of Ordered Set Indication (OSI) signal. 
The 8 Khz timing signal is carried on these same Fiber Channel finks and local drop busses via the supercel Is. The two 
Timing Ordered Sets, ETOS and STOS (Table 0, are used to distribute ftning information through-out the system, as 
discussed in greater detail below. They may therefore occur at any time, even in the middle of a supercell or ATM ce!L 
[0142] Each switching ASIC (FJgure 7) Is capable of generating and outputtlng a continuous stream of supercede. 
This data stream consists only of ordered sets end proprietary formatted 60-byte cells. Cells that are received by a 
switching ASIC tor transmission are Inserted into a ceO slot within this output stream of eupercells. When a 60-byte for- 
matted ATM cell is not available for transmission, either an empty cell or one or more Idle Ordered Sate are Inserted, 
since both represent unused bandwidth. 

[0148] As discussed above, since some Interface cards 21 require a standard timing reference, the system of the 
present Invention provides means tor distributing system timing throughout the switching fabric. Any UCS In an periph- 
eral sheff 3A, 38, etc, can contain an Interlace card 21 which acts as a reference dock source (eg. a Tl Interface). If a 
card is chosen to be a reference dock source, it will be enabled to transmit its clock signal to the local hub card 23 via 
a backplane fine designated as ESYNC Ail Interface cards 21 share this One to the hub card 23, and only drive the fine 
If enabled. The ESYNC signal received by hub card 23 is distributed to the rest of the system as an ETOS signal (Table 
I) via the switching fabric The ETOS signal is routed through the system to a System Synchronization Unit (SSU), 
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which uses the received ETOS signal to generate STOS from the ETOS timing reference. The STOS signal la tnen re- 
distributable throughout the system, for receipt of STOS by any card In the system. The SSU receives the ETOS refer- 
ence dock signal via the switch fabric drop bus 27. In this way, the reference clock signal can reach the system syn- 
chronization unit (SSU) regardless of where the SSU Is located. 

6 [9144] Synchronous and asynchronous transmission Interfaces can both be used to provide reference clocks. Syn- 
chronous interfaces Inherently contain reference timing In the Interface's data signal. Asynchronous Interfaces can con- 
tain reference timing tn the form of the PLC? frame rets, which has no relationship to the physical data rate of the 
Interface. An asynchronously provided reference dock usually contains considerable jitter, typically at 8 Khz, but this 
can easily be fatered out by the SSU. Examples of synchronous Interfaces would be E1 and T1;ES and T3 carry refer- 

i o ence timing either asynchronously or synchronously. 

[0145] As discussed above, the system synchronization unit (SSU) is responsible for generating the system dock 
STOS from the selected reference clock ETOS. The SSU is essentially a very stable PLL, coupled with control logic to 
allow selection of different reference sources, and additional logic to minimize system clock perturbations that might 
occur during reference clock changes or failures. The PLL comprises a DAC, VCXO, phase comparator in feedback 

is configuration, in the usual manner. 

[9148) Distribution of the system dock is accomplished via the swttching fabric, providing distribution to all Interface 
cards 21 without requiring a dedcated dock network. The mechanism by which this Is done te the aforementioned 
Ordered Set (OS). As discussed above, an Ordered Set (OS) comprises 32 bits of data which are transmitted on the 
drop bus 27. The OS is uniquely identified by a corresponding Ordered Set Indicator (OS I) pulse. A single bit In the 32 

20 bit data pattern indicates if the OS Is afeo a System Timing OS (STOS), a sped a) case of OS which is the equivalent of 
a rising edge of an B Khz dock pulse. 

[0147] The same mechanism is used by hub cards 23 In order to trans mS the reference clock from an interface card 
2 1 to the SSU In this case a dngle bit In the OS pattern is used to Indicate ff the OS te afeo an ESYNC Timing OS 
(ETOS), which is a spedal case of an OS which Is the equivalent of a rWng edge of an 8 Khz reference clock putea 

25 [0143] In the event that the system and reference dock signals experience simultaneous rising edges, the STOS 
and ETOS must occur simultaneously. This Is possible within a single OS by asserting both the STOS and ETOS bits, 
therefore the switching fabric is capable of distributing multiple dock signals simultaneously. 
[0149) Due to the flexibility of the reference clock and system dock distribution method, the location of the SSU 
within the system Is also very flexible. The SSU must be located within an peripheral shelf 3A, 3B, etc., but there Is no 

so restriction as to which peripheral shelf within the system contains the SSU, unless the configuration matrix is not a non- 
blocking one. The SSU can be located on any hub card 23, or can be located on a dedicated card that la installed in a 
UCSsfeL 

[0160] Distribution of ETOS and/or STOS timing signals through a multi-stage switching core can be accomplished 
In many ways. Figure 16 shows a possible distribution scenario In the 32x32 switching core 1 (depicted here in stages 
3s 1 A, 1 B, etc). Note that It Is sufficient tor only one of the iSLs to carry TOSs out of the first and second stages 1 A and 
1B of the switch. 

[0151] Modifications and alternative embodiments of the Invention are possible within the sphere and scope of the 
Invention as described herein. 

40 Claims 

1 . A communication system comprising: 

a) a plurality of interconnected universal card slots; 
45 b) a plurality of interface cards and control cards Insertabie Into said universal card slots for receiving Incoming 

ATM cells containing routing information and either date or control and diagnostic information, respectively, and 
transmitting outgoing ones of said ATM cells containing routing information and either data or control and diag- 
nostic Information, respectively ; 

c) means within each of said interface cards and control cards for prepending to said outgoing ones of said 
50 ATM oeifs a plurality of adcBtional header bytes to provide muftiple system functions In addition to ATM cefl rout- 
ing in accordance with said routing Information; and 

d) means for receiving said outgoing ones of said ATM cells, filtering said additional header bytes and in 
response implementing predetermined ones of said multiple system functions. 

55 2. The communication system of claim 1 . wherein one of said multiple system functions comprises selective poirtMo- 
pcint or polnt-to-mu Wpomt routing of said outgoing ones of said ATM ceils within said system. 

3. The communication system of claim 1, wherein one of said multiple system functions comprises In-band control 
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and diagnostics. 

4. The communication system of claim 1 , wherein one of said multiple system functions comprises priority queuing of 
said ATM cells. 

5. The communication system of clatn 1 , wherein one of said multiple system functions comprises sfrnplffled ATM ceil 

statistics gathering. 

6. The communication system of claim 1 , wherein one of said multiple system functions comprises error detection 
across said ATM cells and pre-pended additional header bytes. 

7. A communteatfcn system comprising: 

a) a plurafity of peripheral access shelves each comprising a plurality of Interconnected universal card slots; 

b) a plurality of inters hefl links for Interconnecting said peripheral access shelves; 

c) a plurality of common equipment cards Including a plurality of interface cards InsertaWe Into any one of said 
pluraffiy of universal card slots tor receiving Incoming ATM cells containing data, a first predetermined one of 
said interface cards being provided tor generating a reference clock signal (ESYNC), and a predetermined one 
of said common equipment cards for generating an extracted timing signal (ETOS) derived from said reference 

™~ ctocksignaf(ESYNC); 

d) a system dining unit (SSU) insertaHe into arty one of said plurality of universal card slots for receiving said 
extracted timing signal (ETOS) and irt response generating a system timing signal (STOS); 

e) means within each of said peripheral shelves tor transmitting and receiving supercells via said Intershelf 
links, each of said supercede comprising a Start of Supercefl (SOS) Ordered Set and a predetermined number 
of said outgoing ATM cells; and 

f) mem within each of said peripheral shelves for transmitting and receiving Ordered Sets, wherein each of 
said Ordered Sets comprises a first data byte in the form of a symbol character for Identifying each of said 
Ordered Sets and three further data bytes, at least one of said farther data bytes Including predetermined bits 
for encoding a start of superset! Indicator, for encoding said extracted liming signal (ETOS) derived from said 
reference clock signal (ESYNC) and for encoding said system dining signal (STOS), end wherein each of said 
Ordered Sets Is insertabie Into any location within said supercells, Independent of ATM ceO boundaries. 

8. The communication system of dalm 7, wherein said symbol character comprises a K28.6 character byte In 80/108 
encoding. 

9l The communication system of claim 7, wherein one of said further data bytes comprises a Drive & Scan byte for 
effecting out-of-band communication between said peripheral shelves. 

10. The communication system of claim 7, wherein said reference dock signal (ESYNC) is transmitted to said prede- 
termined one of said common equipment cards along a common backplane bus. 

11. TFftecemmunlcation system of dalm 7, wherein said predetermined one of said common equipment cards transmits 
said extracted timing signal (ETOS) along a drop bus of said communication system. 

12. The communication system of claim 7, wherein said predetermined one of sakj common equipment cards further 
comprises a hub card. 

13. In a communication system for transmitting and receiving data celts containing control messaging and customer 
data, a real-time development system (RDS) comprising: 

a) means for generating a plurality of RDS events end RDS commands at predetermined locations within said 
communication system, each of said RDS events Including an event Werrtffier and a variable length sequence 
of data bytes for defining said event, and each of said RDS commands tnduting a command Identifier and a 
variable length Identifier for defining said command; 

b) means for transmitting said RDS events and RDS commands as data cells throughout said communication 
system via Identical communication links; 

c) means located throughout said communication system for selectively Uttering said RDS events and RDS 
commands from said control messaging data and customer data; 
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d) means located throughout said communication system tor cfis playing said RDS events which have bean fil- 
tered for real-time diagnostics; and 

e) moans tor supporting source and assembly level debug operations using said BOS commands and RDS 
events. 

14. The real-time development system (RDS) of claim 13, wherein said means for generating further comprises means 
for pre-pendlng predetermined header bytes to said sequence of date bytes for Distinguishing said RDS events 
from saw control messaging data and customer data 

m The reed-frne development system (RDS) of cteVn 13, further comprising means located throughout said commu- 
nication system for distinguishing oflforent types of said RDS events based on said event Identifier and said variable 
length sequence of data bytes. 

16- The real-time development system (RDS) of darn 13, wherein said means for displaying further comprises one of 
either a VT100 terminal or workstation connected locally to said communication system. 

17. The real-time development system (RDS) of claim 1 3, wherein said plurality of header bytes contain data for routing 
said RDS events to said predetermined locations Within said communication system. 

13. The real-time development system (RDS) of claim 13, wherein said means tor selectively filtering said RDS events 
further comprises a control card for providing remote access to said communication system. 

19. The reaJ-tlne development system (RDS) of claim 13, wherein said debug operations are implemented vta prede- 
termined ATM cells transmitted between a host processor for generating said RDS commands and a target proc- 
essor for receiving said RDS commands and In response generating RDS events. 
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